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State Use of ACT—Peer Review Results 


In order to comply with the regulations put forth by the Elementary and Secondary Education Act (ESEA), 
as amended by the Every Student Succeeds Act (ESSA), states are required to submit evidence associated 


with their statewide accountability systems to the United States Department of Education (ED) for peer 


review. Thirty-eight states submitted evidence for their assessment systems to the peer review process 


in 2016. As of March 2017, ED has issued thirty-six decision letters in response. Of those responses, two 
states, Wisconsin and Wyoming, received feedback regarding use of the ACT as their high school reading/ 


language arts and mathematics assessments for accountability purposes. The Center on Standards and 
Assessment Implementation (CSAI) was interested in looking at the feedback provided to states using the 
ACT in state plans, and if utilization of this assessment for accountability purposes meets ED requirements. 
The information included in this brief outlines the findings of ED’s feedback to Wisconsin and Wyoming 
contained in the decision letters sent to each state regarding their use of the ACT. 
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Introduction 

The assessment peer review process mandated by ESEA aims to ensure that states implement rigorous academic 
standards and high-quality assessments. The peer review process is evidence-based, drawn from documentation 
submitted by states regarding the process used to develop and administer the assessments, and data to confirm the 
quality of the assessment system. The review is conducted by cohorts of external assessment experts. Since ED paused 
peer review in 2012, almost every state has (a) substantially changed its academic content standards or adopted new 
ones and (b) adopted new or revised assessments associated with those standards. As a result, all states must submit 
documentation for peer review of reading/language arts, mathematics, and science general and alternate assessments. 


ED Peer Review Guidance Document! 
In the ED Peer Review Guidance (Guidance) document issued in 2015, ED has broken down the type of evidence states 
must submit into 30 Critical Elements (CEs), grouped into six sections: 


. Statewide system of standards and assessments 
. Assessment system operations 

. Technical quality - validity 

. Technical quality - other 

. Inclusion of all students 


No fFwWwDNS 


. Academic achievement standards and reporting 


¢ The Guidance document lists each individual CE and includes a description of the level of quality states must 
meet to meet the requirements of the ED. 

¢ Examples of the types of evidence states may choose to include in their peer review information packet are 
also included for each CE. 

2 If astate’s assessment is technology based, specific examples of evidence are provided as well. 

¢ The list of evidence included in the Guidance is not exhaustive, and states may choose to submit types of 
evidence not listed. 

¢ The Guidance also illustrates which CEs are likely to be addressed by state-specific evidence, which are likely 
to be addressed by evidence that can be coordinated among states that administer the same assessments, 
which may be addressed with a combination of state-specific and coordinated evidence, and which CEs will 
be checked for completeness by ED. 


After a state submits their evidence packets, assembled as instructed by the Guidance, teams of expert reviewers 
trained by ED then review and submit notes on their findings back to states to provide them with initial feedback. ED 
then makes a final determination as to whether submitted evidence illustrates that a state’s assessment system either 
meets, substantially meets, partially meets, or does not meet requirements. If required, states must then provide the 
additional evidence requested to prove that the assessment system(s) in question meets the requirements of ED. 


1 U.S. Department of Education Peer Review of State Assessment Systems: Non-Regulatory Guidance for States for Meeting Requirements of the Elementary and Secondary Education 
Act of 1965, as amended. (1st ed.) (2015). Washington, DC: U.S. Department of Education. Retrieved from https://www2.ed.gov/policy/elsec/guid/assessguid15.pdf. 
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High School Assessment Component: Case Studies Using Wisconsin and Wyoming 
Looking specifically at the use of the ACT, the assessment was found to partially meet the requirements of ED in both 
Wisconsin and Wyoming. Additional evidence addressing various CEs will need to be submitted for further review. 


Summary 

There are nine CEs for which the Wisconsin Department of Public Instruction (WIDPI) and the Wyoming Department of 
Education (WDE) must both submit additional evidence surrounding their use of the ACT, with the bulk coming from 
CE Section 5: Inclusion of all Students. There are 12 CEs total for which either Wisconsin or Wyoming must submit ACT 
related additional evidence. The nine overlapping CEs are: 


2.1 - Test Design and Development, 

3.1 - Overall Validity, Including Validity Based on Content, 
3.3 - Validity Based on Internal Structure, 

5.1 — Procedures for Including Students with Disabilities, 

5.2 - Procedures for Including ELs, 

5.3 - Accommodations, 

5.4 - Monitoring Test Administration for Special Populations, 
6.2 - Achievement Standards-Setting, and 

6.4 — Reporting. 
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Though the panel may request that WIDPI and WDE submit additional evidence/documentation for the same CEs, the 
specific evidence requested may or may not be the same between the states and may apply to multiple assessments 
(e.g., ACT and the state's alternate assessment). 


An example of a request for the same evidence for both states can be found for CE 5.4 - Monitoring Test Administration 
for Special Populations. For this CE, both states must submit additional evidence related to their processes for 
monitoring the testing of students with disabilities and ELs to ensure that they are appropriately included in 
assessments and receive the appropriate accommodations. 


However, for CE 5.1 — Procedures for Including Students with Disabilities, the states are asked to submit documentation 
that addresses different areas of concern. For this CE, WIDPI is required to submit “evidence that clarifies what 
specific accessibility tools are available to all students, including students with disabilities, taking the ACT 
tests.” For the same CE, WDE is asked to submit two different types of documentation within the context of the 

ACT, including “documentation of training provided to teachers on accommodations for the .. . ACT,” and 
“documentation to clarify the relationship between WDE’s inclusion/accommodation policies and ACT’s.” 
Though the same CE was found not to be sufficiently addressed for both states, the additional evidence each state 
needs to provide in order to meet the requirements of the CE is different. 
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In some cases, the requests for additional evidence concerning the same CE may be worded differently in each letter, 
but ask for evidence that addresses a similar requirement found to be lacking in the evidence submitted for each state. 
To meet the requirements of CE 3.3 - Validity Based on Internal Structure, WDE is required to submit “[confirmatory] 
evidence that the reporting structures of the ... ACT are consistent with the sub-domain structures of the 
State’s academic content standards.” WIDPI must provide “evidence that establishes the ACT reading, English 
and writing tests as a single R/LA construct.” The panel then goes on to list the types of evidence the state may 
include in their resubmission: 


¢ Reports of analyses of the internal structure of the assessments (e.g., tables of item correlations) that show 
the extent to which the interrelationships among subscores are consistent with the State’s academic content 
standards for relevant student groups; 


OR 


¢ Reports of analyses that show the dimensionality of the assessment is consistent with the structure of the 
State’s academic content standards and the intended interpretations of results; 


OR 
¢ Evidence that ancillary constructs needed for success on the assessments do not provide inappropriate 


barriers for measuring the achievement of all students, such as evidence from cognitive labs or 
documentation of item development procedures; 
OR 

¢ Reports of differential item functioning (DIF) analyses that show whether particular items (e.g., essays, 
performance tasks, or items requiring specific knowledge or skills) function differently for relevant student 
groups 


While the information requested was stated differently to each state education agency, both states are required to 
undergo similar activities to gather the evidence needed to meet ED requirements. Both states must acquire and 

then provide confirmatory evidence from the test publisher that shows an analysis of the internal structure of the 
assessment and its interrelationships as related to either psychometrics (WIDPI) or the measurement (WDE) of the final 
assessment results as related to state standards. 


Another detail present in the feedback provided to the states is that the panels specified whether the additional 
evidence requested applies to the entire assessment system for the state, only the ACT, or only the R/LA portion of 
the ACT. 


A side-by-side comparison of all overlapping Critical Elements and feedback that addresses the additional evidence 
WIDPI and WDE must submit surrounding their use of the ACT are included in Appendices A-C. 
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The following charts list the Critical Elements for which WIDPI and/or WDE must submit additional evidence with regard to their use of the ACT for their state’s 
reading/language arts and mathematics assessment for accountability purposes. Feedback may be inclusive of other state-specific assessments. 


Appendix A: Critical elements for which both WIDPI and WDE must submit additional evidence 


CRITICAL ELEMENT WISCONSIN WYOMING 


- Forall assessments in reading/language arts, evidence 
regarding how the Wyoming Department of Education 


For the reading/language arts (R/LA) and mathematics [W/ feedback 
contains verbiage that notes which portion(s) of the ACT (reading and 
language arts, mathematics) the state is required to submit additional (WDE) plans to assess the full breadth of its reading/ 
evidence for] general assessments in high school (ACT), WIDPI must language arts standards, including writing, speaking and 
provide: listening. [NOTE: Wyoming has received a speaking and 
listening waiver; therefore, the Department does not 


2.1 - Test Design and Development 


The State's test design and test development process is well-suited 
for the content, is technically sound, aligns the assessments to the 
full range of the State’s academic content standards, and includes: 


- Statement(s) of the purposes of the assessments and the 


intended interpretations and uses of results; — Evidence that the test design measures the full range of the State's 


- Test blueprints that describe the structure of each assessment 
in sufficient detail to support the development of assessments 
that are technically sound, measure the full range of the State's 
grade-level academic content standards, and support the 


grade-level academic content standards (e.g., evidence of alignment 
of the test design blueprint to academic content standards). This 
evidence should include information about the State's plan to assess 
the full breadth of the State’s R/LA standards, including speaking and 
listening. (Note: WIDPI has received a speaking and listening waiver; 


expect Wyoming to submit additional evidence regarding 
speaking and listening during the period of the waiver.] 
Documentation of independent alignment studies 


between the State’s academic content standards and the 
State's high school assessment, the ACT. 


intended interpretations and uses of the results; ‘A = 
s therefore, the Department does not expect WIDPI to submit additional 


— Processes to ensure that each assessment is tailored to the evidence regarding speaking and listening during the period of the 
knowledge and skills included in the State's academic content waiver.) 


standards, reflects appropriate inclusion of challenging content, 
and requires complex demonstrations or applications of 
knowledge and skills (i.e., higher-order thinking skills); 


- Evidence for the R/LA tests that describe the use of writing and reading 
test scores to support the intended interpretations and use of the 
results for R/LA accountability purposes. 


— If the State administers computer-adaptive assessments, the 
item pool and item selection procedures adequately support 
the test design. 
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CRITICAL ELEMENT WISCONSIN WYOMING 


- Evidence of independent alignment studies evaluating 
the test items to the State content standards for all 
assessments (PAWS, WyAlt, and ACT). [WY feedback 
includes all state assessments for which the state will have 


For the R/LA and mathematics general assessments in high school (ACT), 
WIDPI must provide: 


3.1 - Overall Validity, including Validity Based on Content 


The State has documented adequate overall validity evidence for 
its assessments, and the State's validity evidence includes evidence 
that the State’s assessments measure the knowledge and skills 


- Evidence that independently establishes alignment, specifically that: 


¢ Each assessment is aligned to its test blueprint, and each blueprint is 


specified in the State’s academic content standards, including: 


- Documentation of adequate alignment between the State's 
assessments and the academic content standards the 
assessments are designed to measure in terms of content 
(i.e., knowledge and process), the full range of the State's 
academic content standards, balance of content, and cognitive 
complexity; 


- Ifthe State administers alternate assessments based on 
alternate academic achievement standards, the assessments 


aligned to the full range of the State’s academic content standards, 
including speaking and listening in R/LA (Note: WIDPI has received a 
speaking and listening waiver; therefore, the Department does not 
expect WIDPI to submit additional evidence regarding speaking and 
listening during the period of the waiver); or 


Each assessment is aligned to the full range of the State’s academic 
content standards, and the procedures the State follows to ensure 
such alignment during test development; and 


Describes a systematic process and timeline to address any gaps or 


to provide additional evidence, including ACT. PAWS refers 
to the Proficiency Assessment for Wyoming Students, 
administered in grades 3-8 in reading and math, and 

grades 4 and 8 in science. Wy-ALT refers to the Wyoming 
Alternate Assessment for Students with Significant Cognitive 
Disabilities, the state's alternative assessment.] 


show adequate linkage to the State’s academic content 
standards in terms of content match (i.e., no unrelated 
content) and the breadth of content and cognitive complexity 
determined in test design to be appropriate for students with 
the most significant cognitive disabilities. 


weaknesses identified through analysis of alignment. 


-— See evidence in 2.1 above regarding the use of writing and reading test 
scores to support the intended interpretations and use of the results 
for R/LA accountability purposes. 


For the R/LA general assessments in high school (ACT), WIDPI must - Evidence that the reporting structures of the PAWS, 
WyAlt, and ACT are consistent with the sub-domain 
structures of the State's academic content standards. 
Evidence for this critical element is typically confirmatory, 
but what is provided is procedural. 


3.3 - Validity Based on Internal Structure 


The State has documented adequate validity evidence that the provide: 


scoring and reporting structures of its assessments are consistent 
with the sub-domain structures of the State’s academic content 
standards on which the intended interpretations and uses of 
results are based. 


- Evidence that establishes the ACT reading, English, and writing tests as 
a single R/LA construct. Evidence may include: 


¢ Reports of analyses of the internal structure of the assessments 
(e.g., tables of item correlations) that show the extent to which the 
interrelationships among subscores are consistent with the State's 
academic content standards for relevant student groups; OR 


¢ Reports of analyses that show the dimensionality of the assessment 
is consistent with the structure of the State’s academic content 
standards and the intended interpretations of results; OR 


¢ Evidence that ancillary constructs needed for success on the 
assessments do not provide inappropriate barriers for measuring 
the achievement of all students, such as evidence from cognitive 
labs or documentation of item development procedures; OR 


¢ Reports of differential item functioning (DIF) analyses that show 
whether particular items (e.g., essays, performance tasks, or items 
requiring specific knowledge or skills) function differently for 
relevant student groups. 
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CRITICAL ELEMENT WISCONSIN WYOMING 


5.1 - Procedures for Including Students with Disabilities For the R/LA and mathematics general assessments in high school (ACT), — Documentation of training provided to teachers on 

The State has in place procedures to ensure the inclusion of all WIDPI must provide: accommodations for the PAWS and ACT. 

public elementary and secondary school students with disabilities — Evidence that clarifies what specific accessibility tools are available to — Documentation to clarify the relationship between WDE's 
in the State’s assessment system, including, at a minimum, all students, including students with disabilities, taking the ACT tests. inclusion/accommodation policies and ACT's. 


guidance for individual educational plan (IEP) Teams to inform 
decisions about student assessments that: 


- Provides clear explanations of the differences between 
assessments based on grade-level academic achievement 
standards and assessments based on alternate academic 
achievement standards, including any effects of State and 
local policies on a student's education resulting from taking an 
alternate assessment based on alternate academic achievement 
standards; 


- States that decisions about how to assess students with 
disabilities must be made by a student's IEP Team based on each 
student's individual needs; 


- Provides guidelines for determining whether to assess a student 
on the general assessment without accommodation(s), the 
general assessment with accommodation(s), or an alternate 
assessment; 


- Provides information on accessibility tools and features available 
to students in general and assessment accommodations 
available for students with disabilities; 


- Provides guidance regarding selection of appropriate 
accommodations for students with disabilities; 


- Includes instructions that students eligible to be assessed based 
on alternate academic achievement standards may be from any 
of the disability categories listed in the IDEA; 


- Ensures that parents of students with the most significant 
cognitive disabilities are informed that their student's 
achievement will be based on alternate academic achievement 
standards and of any possible consequences of taking the 
alternate assessments resulting from district or State policy (e.g., 
ineligibility for a regular high school diploma if the student does 
not demonstrate proficiency in the content area on the State’s 
general assessments); 


- The State has procedures in place to ensure that its 
implementation of alternate academic achievement standards 
for students with the most significant cognitive disabilities 
promotes student access to the general curriculum. 
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CRITICAL ELEMENT WISCONSIN WYOMING 


5.2 — Procedures for Including ELs 


The State has in place procedures to ensure the inclusion of all 
English learners in public elementary and secondary schools in 
the State’s assessment system and clearly communicates this 
information to districts, schools, teachers, and parents, including, 
at a minimum: 


— Procedures for determining whether an English learner should 
be assessed with accommodation(s); 


- Information on accessibility tools and features available to all 
students and assessment accommodations available for English 
learners; 


- Guidance regarding selection of appropriate accommodations 
for English learners. 


5.3 - Accommodations 


The State makes available appropriate accommodations and 
ensures that its assessments are accessible to students with 
disabilities and English learners. Specifically, the State: 


- Ensures that appropriate accommodations are available for 
students with disabilities under the Individuals with Disabilities 
Education Act (IDEA) and students covered by Section 504; 


- Ensures that appropriate accommodations are available for 
English learners; 


- Has determined that the accommodations it provides (i) are 
appropriate and effective for meeting the individual student's 
need(s) to participate in the assessments, (ii) do not alter 
the construct being assessed, and (iii) allow meaningful 
interpretations of results and comparison of scores for students 
who need and receive accommodations and students who do 
not need and do not receive accommodations; 


- Has a process to individually review and allow exceptional 
requests for a small number of students who require 
accommodations beyond those routinely allowed. 


For the R/LA and mathematics general assessments in high school (ACT), 


WIDPI must provide: 


- As noted in element 5.1, evidence that clarifies what specific 


accessibility tools are available to all students, including ELs, taking the 


ACT tests. 


For the R/LA and mathematics general assessments in high school (ACT), 


WIDPI must provide: 


- Evidence of a process to determine that the accommodations it 
provides (i) are appropriate and effective for meeting the individual 


student's need(s) to participate in the assessments, (ii) do not alter the 
construct being assessed, and (iii) allow meaningful interpretations of 


results and comparison of scores for students who need and receive 
accommodations and students who do not need and do not receive 
accommodations. 


Evidence of training for teachers on English learner 
accommodations for PAWS and ACT. 


Evidence that English learners receive appropriate 
accommodations related to their English proficiency 
based on their individual needs. 


Documentation of the process, including who is on the 
decision-making team, for determining accommodations 
for English learners. 

Evidence of a decision-making framework that schools 
use to make accommodations decisions. 


Documentation of the process for reviewing an 
exceptional accommodation request (PAWS, WyAlt, ACT). 
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CRITICAL ELEMENT WISCONSIN WYOMING 


5.4 - Monitoring Test Administration for Special Populations For the R/LA and mathematics general assessments in high school — Evidence that WDE monitors test administration in 


The State monitors test administration in its districts and schools to 
ensure that appropriate assessments, with or without appropriate 
accommodations, are selected for students with disabilities under 
IDEA, students covered by Section 504, and English learners so 
that they are appropriately included in assessments and receive 
accommodations that are: 


- Consistent with the State's policies for accommodations; 


— Appropriate for addressing a student's disability or language 
needs for each assessment administered; 

— Consistent with accommodations provided to the students 
during instruction and/or practice; 


— Consistent with the assessment accommodations identified by a 
student's IEP Team or 504 team for students with disabilities, or 
another process for an English learner; 


- Administered with fidelity to test administration procedures. 


(ACT)...WIDPI must provide: 


- Evidence of a process for monitoring testing of students with 
disabilities and ELs to ensure that they are appropriately included in 
assessments and receive accommodations that are: 


° 


od 


Consistent with the State's policies for accommodations; 


Appropriate for addressing a student's disability or language needs 
for each assessment administered; 


Consistent with accommodations provided to the students during 
instruction and/or practice; 


Consistent with the assessment accommodations identified by a 
student's IEP Team or 504 team for students with disabilities, or 
another process for an EL; and 


Administered with fidelity to test administration procedures 


its districts and schools to ensure that appropriate 
assessments, with or without appropriate 
accommodations, are selected for students with 
disabilities under the Individuals with Disabilities 
Education Act, students covered by Section 504, and 
English learners so that they are appropriately included in 
assessments and receive accommodations that are: 


* Consistent with the State's policies for 
accommodations; 


¢ Appropriate for addressing a student's disability or 
language needs for each assessment administered; 


¢ Consistent with accommodations provided to the 
students during instruction and/or practice; 


¢ Consistent with the assessment accommodations 
identified by a student's individualized educational 
program team or 504 team for students with 
disabilities, or another process for an English learner; 


¢ Administered with fidelity to test administration 
procedures (PAWS, WyAlt, ACT). 
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CRITICAL ELEMENT WISCONSIN WYOMING 


6.2 - Achievement Standards-Setting For the R/LA and mathematics general assessments in high school (ACT), - Clarify how the WDE performance level descriptors (PLDs) 
The State used a technically sound method and process that WIDPI must provide: for the ACT are connected to WDE's content standards 
involved panelists with appropriate experience and expertise - Evidence that provides greater detail about the achievement standard and to the interpretations of academic knowledge and 
for setting its academic achievement standards and alternate setting process, including: skills. 

academic achievement standards to ensure they are valid and 


¢ Adescription of the standards-setting method and process used by 


reliable. the State; 


¢ The rationale for the method selected; 


¢ Documentation that the method used for setting cut scores allowed 
panelists to apply their knowledge and experience in a reasonable 
manner and supported the establishment of reasonable and 
defensible cut scores; 


¢ Documentation of the process used for setting cut scores and 
developing performance-level descriptors aligned to the State's 
academic content standards; 


¢ Adescription of the process for selecting panelists; 


¢ Documentation that the standards-setting panels consisted of 
panelists with appropriate experience and expertise, including: 


* Content experts with experience teaching the State’s academic 
content standards in the tested grades; 


Individuals with experience and expertise teaching students with 
disabilities, English learners and other student populations in the 
State; 


As appropriate, individuals from institutions of higher education 
and individuals knowledgeable about career-readiness; and 


A description, by relevant characteristics, of the panelists (overall 
and by individual panels) who participated in achievement 
standards setting. 
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CRITICAL ELEMENT WISCONSIN WYOMING 


6.4 - Reporting 


The State reports its assessment results, and the reporting 
facilitates timely, appropriate, credible, and defensible 
interpretations and uses of results for students tested by parents, 
educators, State officials, policymakers and other stakeholders, and 
the public, including: 


— The State reports to the public its assessment results on student 
achievement at each proficiency level and the percentage of 
students not tested for all students and each student group 
after each test administration; 


— The State reports assessment results, including itemized score 
analyses, to districts and schools so that parents, teachers, 
principals, and administrators can interpret the results and 
address the specific academic needs of students, and the State 
also provides interpretive guides to support appropriate uses of 
the assessment results; 


— The State provides for the production and delivery of individual 
student interpretive, descriptive, and diagnostic reports after 
each administration of its assessments that: 


¢ Provide valid and reliable information regarding a student's 
achievement; 


¢ Report the student’s achievement in terms of the State's 
grade-level academic achievement standards (including 
performance-level descriptors); 


¢ Provide information to help parents, teachers, and principals 
interpret the test results and address the specific academic 
needs of students; 


¢ Are available in alternate formats (e.g., Braille or large print) 
upon request and, to the extent practicable, in a native 
language that parents can understand; 


¢ The State follows a process and timeline for delivering 
individual student reports to parents, teachers, and principals 
as soon as practicable after each test administration. 


For the R/LA and mathematics general assessments in high school (ACT), 
WIDPI must provide: 


Evidence that test score reports and supporting material (a) reflect the 
State's test reporting categories (i.e., a single score for R/LA), and (b) 
provide information on the State’s academic achievement levels. 


Evidence that the score reports are available in alternative formats. 


Evidence of a process and timeline for delivering reports to students, 
parents, teachers, principals, and other stakeholders as soon as 
practicable after each test administration. 


Evidence of materials that support parents and educators in the use 
and interpretation of test scores. 


- Documentation of the process and timeline for delivering 


individual score reports for PAWS, WyAlt, and ACT. 
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Appendix B: Additional critical elements for which WIDPI must submit additional evidence 


CRITICAL ELEMENT ADDITIONAL EVIDENCE REQUESTED (FOR ACT) 


3.4 - Validity Based on Relationships with Other Variables For the R/LA and mathematics general assessments in high school (ACT), WIDPI must provide: 

The State has documented adequate validity evidence that the State’s assessment scores are related as - Evidence that shows the assessment scores are related as expected with criterion and other variables 

expected with other variables. for all student groups (e.g., reports of analyses that demonstrate positive correlations between State 
assessment results and assessments of the same content area administered by some or all districts in 
the State). 

4.1 - Reliability For the R/LA general assessments in high school (ACT), WIDPI must provide: 

The State has documented adequate reliability evidence for its assessments for the following measures — Evidence that supports the reliability for the composite R/LA test scores using the State's data from 

of reliability for the State’s student population overall and each student group and, if the State's test administration, such as: 

assessments are implemented in multiple States, for the assessment overall and each student group, 


¢ Reliability estimates for the State overall and major reporting sub-groups. 
including: : - 
¢ Standard error of measurement for the State overall and major reporting sub-groups. 


Test reliability of the State’s assessments estimated for its student population; ; j : i 
¢ Estimates of classification accuracy and decision consistency for the State overall and major 


Overall and conditional standard error of measurement of the State’s assessments; reporting sub-groups 


- Consistency and accuracy of estimates in categorical classification decisions for the cut scores and 
achievement levels based on the assessment results; 


- For computer-adaptive tests, evidence that the assessments produce test forms with adequately 
precise estimates of a student’s achievement. 


4.4 - Scoring For the R/LA general assessments in high school (ACT), WIDPI must provide: 
The State has established and documented standardized scoring procedures and protocols for its - Evidence on the hand scoring for the writing tests, such as rater recruiting criteria, training, range 
assessments that are designed to produce reliable results, facilitate valid score interpretations, and finding/calibration, validity papers, and procedures to reconcile discrepant ratings among human 


report assessment results in terms of the State’s academic achievement standards. scorers. 


State Use of ACT—Peer Review Results 


CRITICAL ELEMENT ADDITIONAL EVIDENCE REQUESTED (FOR ACT) 


4.7 - Technical Analysis and Ongoing Maintenance For the R/LA and mathematics general assessments in high school (ACT), WIDPI must provide: 

The State has a system for monitoring and maintaining, and improving as needed, the quality of its — Evidence of regular internal and external technical review of the ACT testing program in the State, 
assessment system, including clear and technically sound criteria for the analyses of all the assessments such as minutes from technical advisory committee (TAC) meetings and documentation of roles and 
in its assessment system (i.e., general assessments and alternate assessments). responsibilities of TAC members. 

6.3 - Challenging and Aligned Academic Achievement Standards For the R/LA and mathematics general assessments in high school (ACT), WIDPI must provide: 

The State’s academic achievement standards are challenging and aligned with the State’s academic - Evidence that the State’s academic achievement standards are aligned with the State’s academic 
content standards such that a high school student who scores at the proficient or above level has content standards. 


mastered what students are expected to know and be able to do by the time they graduate from high 


; ‘ — Evidence that the State’s academic achievement standards are challenging. 
school in order to succeed in college and the workforce. 


If the State has defined alternate academic achievement standards for students with the most 
significant cognitive disabilities, the alternate academic achievement standards are linked to the 
State’s grade-level academic content standards or extended academic content standards, show linkage 
to different content across grades, and reflect professional judgment of the highest achievement 
standards possible for students with the most significant cognitive disabilities. 
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Appendix C: Additional critical elements for which WDE must submit additional evidence 


CRITICAL ELEMENT ADDITIONAL EVIDENCE REQUESTED (FOR AC 


2.3 - Test Administration — Evidence of State policies for test administration to address unexpected or 


The State implements policies and procedures for standardized test administration; specifically the State: irregular testing situations that may arise during course of test administration. 


- Has established and communicates to educators clear, thorough and consistent standardized procedures for the 
administration of its assessments, including administration with accommodations; 


- Has established procedures to ensure that all individuals responsible for administering the State’s general and 
alternate assessments receive training on the State’s established procedures for the administration of its assessments; 


— If the State administers technology-based assessments, the State has defined technology and other related 
requirements, included technology-based test administration in its standardized procedures for test administration, 
and established contingency plans to address possible technology challenges during test administration. 


2.4 - Monitoring Test Administration - Documentation that clearly identifies or clarifies WDE's role in the monitoring of 


The State adequately monitors the administration of its State assessments to ensure that standardized test the ACT test administration. 
administration procedures are implemented with fidelity across districts and schools.. 


2.5 - Test Security - Documentation outlining what remediation WDE requires following test security 
The State has implemented and documented an appropriate set of policies and procedures to prevent test irregularities violations (for all assessments). 

and ensure the integrity of test results through: - Documentation of a test security agreement with ACT, including procedures ACT 
- Prevention of any assessment irregularities, including maintaining the security of test materials, proper test follows to report incidents to WDE. 


preparation guidelines and administration procedures, incident-reporting procedures, consequences for confirmed 
violations of test security, and requirements for annual training at the district and school levels for all individuals 
involved in test administration; 


- Detection of test irregularities; 
- Remediation following any test security incidents involving any of the State's assessments; 


- Investigation of alleged or factual test irregularities. 


2.6 - Systems for Protecting Data Integrity and Privacy — Evidence of a specific data security agreement between WDE and ACT. 


The State has policies and procedures in place to protect the integrity and confidentiality of its test materials, test-related 
data, and personally identifiable information, specifically: 


- To protect the integrity of its test materials and related data in test developments, administration, and storage and use 
of results; 


- To secure student-level assessment data and protect student privacy and confidentiality, including guidelines for 
districts and schools; 


- To protect personally identifiable information about any individual student in reporting, including defining the 
minimum number of students necessary to allow reporting of scores for all students and student groups. 
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CRITICAL ELEMENT ADDITIONAL EVIDENCE REQUESTED (FOR ACT) 


3.2 - Validity Based on Cognitive Processes — Documentation that all of the State’s assessments...measure the intended 

The State has documented adequate validity evidence that its assessments tap the intended cognitive processes cognitive processes appropriate for each grade level as represented in the State's 
appropriate for each grade level as represented in the State’s academic content standards. academic content standards. 

4.2 - Fairness and Accessibility — Documentation on the accommodations that can be used for each assessment, 
The State has taken reasonable and appropriate steps to ensure that its assessments are accessible to all students and fair grade, and subject area (i.e., ... ACT). 

across student groups in the design, development and analysis of its assessments. — Training materials given to ACT fairness review committees and item writers, 


including demographic information on these reviewers. 


6.1 - State Adoption of Academic Achievement Standards for All Students — Documentation of formal adoption of...academic achievement standards for high 


The State formally adopted challenging academic achievement standards in reading/language arts, mathematics and in school (for the ACT). 
science for all students, specifically: 


— The State formally adopted academic achievement standards in the required tested grades and, at its option, also 
alternate academic achievement standards for students with the most significant cognitive disabilities; 


- The State applies its grade-level academic achievement standards to all public elementary and secondary school 
students enrolled in the grade to which they apply, with the exception of students with the most significant cognitive 
disabilities to whom alternate academic achievement standards may apply; 


— The State's academic achievement standards and, as applicable, alternate academic achievement standards, 
include: (a) At least three levels of achievement, with two for high achievement and a third of lower achievement; 
(b) descriptions of the competencies associated with each achievement level; and (c) achievement scores that 
differentiate among the achievement levels. 


CSAI Update is produced by the The Center on Standards and Assessment oy) 
Implementation (CSAI). CSAI, a collaboration between WestEd and CRESST, provides W est&cl * 
state education agencies (SEAs) and Regional Comprehensive Centers (RCCs) with WestEd is a nonpartisan, nonprofit research, development, and service agency that partners with education and other 


research support, technical assistance, tools, and other resources to help inform 
decisions about standards, assessment, and accountability. Visit www.csai-online.org for 
more information. 


communities throughout the United States and abroad to promote excellence, achieve equity, and improve learning for 
children, youth, and adults. WestEd has more than a dozen offices nationwide, from Massachusetts, Vermont and Georgia, to 
Illinois, Arizona and California, with headquarters in San Francisco. 
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expressed herein are those of the author(s) and do not reflect the positions or policies of the U.S. Department of Education. San Francisco, CA 94107-1242. 


